Analyzing Complex Problem Solving by Dynamic Brain Networks.

Complex problem solving is a high level cognitive task of the human brain, which has been studied over the last decade. Tower of London (TOL) is a game that has been widely used to study complex problem solving. In this paper, we aim to explore the underlying cognitive network structure among anatomical regions of complex problem solving and its subtasks, namely planning and execution. A new computational model for estimating a brain network at each time instant of fMRI recordings is proposed. The suggested method models the brain network as an Artificial Neural Network, where the weights correspond to the relationships among the brain anatomic regions. The first step of the model is preprocessing that manages to decrease the spatial redundancy while increasing the temporal resolution of the fMRI recordings. Then, dynamic brain networks are estimated using the preprocessed fMRI signal to train the Artificial Neural Network. The properties of the estimated brain networks are studied in order to identify regions of interest, such as hubs and subgroups of densely connected brain regions. The representation power of the suggested brain network is shown by decoding the planning and execution subtasks of complex problem solving. Our findings are consistent with the previous results of experimental psychology. Furthermore, it is observed that there are more hubs during the planning phase compared to the execution phase, and the clusters are more strongly connected during planning compared to execution.

Gender classification using mesh networks on multiresolution multitask fMRI data.

Brain connectivity networks have been shown to represent gender differences under a number of cognitive tasks. Recently, it has been conjectured that fMRI signals decomposed into different resolutions embed different types of cognitive information. In this paper, we combine multiresolution analysis and connectivity networks to study gender differences under a variety of cognitive tasks, and propose a machine learning framework to discriminate individuals according to their gender. For this purpose, we estimate a set of brain networks, formed at different resolutions while the subjects perform different cognitive tasks. First, we decompose fMRI signals recorded under a sequence of cognitive stimuli into its frequency subbands using Discrete Wavelet Transform (DWT). Next, we represent the fMRI signals by mesh networks formed among the anatomic regions for each task experiment at each subband. The mesh networks are constructed by ensembling a set of local meshes, each of which represents the relationship of an anatomical region as a weighted linear combination of its neighbors. Then, we estimate the edge weights of each mesh by ridge regression. The proposed approach yields 2CL functional mesh networks for each subject, where C is the number of cognitive tasks and L is the number of subband signals obtained after wavelet decomposition. This approach enables one to classify gender under different cognitive tasks and different frequency subbands. The final step of the suggested framework is to fuse the complementary information of the mesh networks for each subject to discriminate the gender. We fuse the information embedded in mesh networks formed for different tasks and resolutions under a three-level fuzzy stacked generalization (FSG) architecture. In this architecture, different layers are responsible for fusion of diverse information obtained from different cognitive tasks and resolutions. In the experimental analyses, we use Human Connectome Project task fMRI dataset. Results reflect that fusing the mesh network representations computed at multiple resolutions for multiple tasks provides the best gender classification accuracy compared to the single subband task mesh networks or fusion of representations obtained using only multitask or only multiresolution data. Besides, mesh edge weights slightly outperform pairwise correlations between regions, and significantly outperform raw fMRI signals. In addition, we analyze the gender discriminative power of mesh edge weights for different tasks and resolutions.

Encoding the local connectivity patterns of fMRI for cognitive task and state classification.

In this work, we propose a novel framework to encode the local connectivity patterns of brain, using Fisher vectors (FV), vector of locally aggregated descriptors (VLAD) and bag-of-words (BoW) methods. We first obtain local descriptors, called mesh arc descriptors (MADs) from fMRI data, by forming local meshes around anatomical regions, and estimating their relationship within a neighborhood. Then, we extract a dictionary of relationships, called brain connectivity dictionary by fitting a generative Gaussian mixture model (GMM) to a set of MADs, and selecting codewords at the mean of each component of the mixture. Codewords represent connectivity patterns among anatomical regions. We also encode MADs by VLAD and BoW methods using k-Means clustering. We classify cognitive tasks using the Human Connectome Project (HCP) task fMRI dataset and cognitive states using the Emotional Memory Retrieval (EMR). We train support vector machines (SVMs) using the encoded MADs. Results demonstrate that, FV encoding of MADs can be successfully employed for classification of cognitive tasks, and outperform VLAD and BoW representations. Moreover, we identify the significant Gaussians in mixture models by computing energy of their corresponding FV parts, and analyze their effect on classification accuracy. Finally, we suggest a new method to visualize the codewords of the learned brain connectivity dictionary.

Hierarchical multi-resolution mesh networks for brain decoding.

Human brain is supposed to process information in multiple frequency bands. Therefore, we can extract diverse information from functional Magnetic Resonance Imaging (fMRI) data by processing it at multiple resolutions. We propose a framework, called Hierarchical Multi-resolution Mesh Networks (HMMNs), which establishes a set of brain networks at multiple resolutions of fMRI signal to represent the underlying cognitive process. Our framework, first, decomposes the fMRI signal into various frequency subbands using wavelet transform. Then, a brain network is formed at each subband by ensembling a set of local meshes. Arc weights of each local mesh are estimated by ridge regression. Finally, adjacency matrices of mesh networks obtained at different subbands are used to train classifiers in an ensemble learning architecture, called fuzzy stacked generalization (FSG). Our decoding performances on Human Connectome Project task-fMRI dataset reflect that HMMNs can successfully discriminate tasks with 99% accuracy, across 808 subjects. Diversity of information embedded in mesh networks of multiple subbands enables the ensemble of classifiers to collaborate with each other for brain decoding. The suggested HMMNs decode the cognitive tasks better than a single classifier applied to any subband. Also mesh networks have a better representation power compared to pairwise correlations or average voxel time series. Moreover, fusion of diverse information using FSG outperforms fusion with majority voting. We conclude that, fMRI data, recorded during a cognitive task, provide diverse information in multi-resolution mesh networks. Our framework fuses this complementary information and boosts the brain decoding performances obtained at individual subbands.

CEREBRA: a 3-D visualization tool for brain network extracted from fMRI data.

In this paper, we introduce a new tool, CEREBRA, to visualize the 3D network of human brain, extracted from the fMRI data. The tool aims to analyze the brain connectivity by representing the selected voxels as the nodes of the network. The edge weights among the voxels are estimated by considering the relationships among the voxel time series. The tool enables the researchers to observe the active brain regions and the interactions among them by using graph theoretic measures, such as, the edge weight and node degree distributions. CEREBRA provides an interactive interface with basic display and editing options for the researchers to study their hypotheses about the connectivity of the brain network. CEREBRA interactively simplifies the network by selecting the active voxels and the most correlated edge weights. The researchers may remove the voxels and edges by using local and global thresholds selected on the window. The built-in graph reduction algorithms are then eliminate the irrelevant regions, voxels and edges and display various properties of the network. The toolbox is capable of space-time representation of the voxel time series and estimated arc weights by using the animated heat maps.

Functional Mesh Model with Temporal Measurements for brain decoding.

We propose a method called Functional Mesh Model with Temporal Measurements (FMM-TM) to estimate a functional relationship among voxels using temporal data, and employ these relationships for brain decoding. For each sample, we measure Blood Oxygenation Level Dependent (BOLD) responses from each voxel, and construct a functional mesh around each voxel (called seed voxel) with its nearest neighbors selected using distance metrics namely Pearson correlation, cosine similarity and Euclidean distance. Then, we represent the BOLD response of a seed voxel in terms of linear combination of BOLD responses of its p-nearest neighbors. The relationship between the seed voxel and its neighbors is represented using a set of weights which are estimated by employing linear regression. We train Support Vector Machine and k-Nearest Neighbor classifiers using the estimated weights as features. We test our model in an event-related design experiment, namely object recognition, and observe that our features perform better than raw voxel intensity values, features obtained using various pairwise distance metrics, and local mesh model features extracted using stationary and temporal measurements.

Ensembling brain regions for brain decoding.

In this study, we propose a new method which ensembles the brain regions for brain decoding. The ensemble is generated by clustering the fMRI images recorded during an experimental set-up which measures the cognitive states associated to semantic categories. Initially, voxel clusters are formed by using hierarchical agglomerative clustering with correlation as the similarity metric. Then, for each voxel cluster, a support vector machine (SVM) classifier is trained to estimate the class-posteriori probabilities. Lastly, the class-posteriori probabilities are ensembled by concatenating them under the same feature space, which are then used to train a meta-layer SVM for the final classification of the cognitive states. By using the voxel clusters, we aim to utilize the distributed, but complementing nature of the semantic representations in the brain and improve the classification accuracy. Thus, we make an existential claim that the brain regions provide a natural basis for ensemble learning which should be superior to the random clusters formed over a selected set of voxels. Our approach yields to better classification accuracies in Mitchell dataset on most of the subjects, when compared to state-of-the-art which emphasizes voxel selection and ensemble learning with random subspaces.

Analyzing the information distribution in the fMRI measurements by estimating the degree of locality.

In this study, we propose a new method for analyzing and representing the distribution of discriminative information for data acquired via functional Magnetic Resonance Imaging (fMRI). For this purpose, we form a spatially local mesh with varying size, around each voxel, called the seed voxel. The relationship among each seed voxel and its neighbors is estimated using a linear regression model by minimizing the square error. Then, we estimate the optimal mesh size that represents the connections among each seed voxel and its surroundings by minimizing Akaike's Final Prediction Error (FPE) with respect to the mesh size. The degree of locality is represented by the optimum mesh size. Our results indicate that the local mesh size with the highest discriminative power varies across individual participants. The proposed method was tested on an fMRI study consisting of item recognition (IR) and judgment of recency (JOR) tasks. For each participant, the estimated arc weights of each local mesh with different mesh size are used to classify the type of memory judgment (i.e.IR or JOR). Classification accuracy for each participant was derived using k-Nearest Neighbor (k-NN) method. The results indicate that the proposed local mesh model with optimal mesh size can successfully represent discriminative information for neuroimaging data.

Representation of cognitive processes using the minimum spanning tree of local meshes.

A new graphical model called Cognitive Process Graph (CPG) is proposed, for classifying cognitive processes based on neural activation patterns which are acquired via functional Magnetic Resonance Imaging (fMRI) in brain. In the CPG, first local meshes are formed around each voxel. Second, the relationships between a voxel and its neighbors in a local mesh, which are estimated by using a linear regression model, are used to form the edges among the voxels (graph nodes) in the CPG. Then, a minimum spanning tree (MST) of the CPG which spans all the voxels in the region of interest is computed. The arc weights of the MST are used to represent the underlying cognitive processes. The proposed method reduces the curse of dimensionality problem that is caused by very large dimension of the feature space of the fMRI measurements, compared to number of instances. Finally, the arc weights computed over the path of the MST called MST-Features (MST-F) are used to train a statistical learning machine. The proposed method is tested on a recognition memory experiment, including data pertaining to encoding and retrieval of words belonging to ten different semantic categories. Two popular classifiers, namely k-Nearest Neighbor (k-NN) and Support Vector Machine (SVM), are trained in order to predict the semantic category of the item being retrieved, based on activation patterns during encoding. The classification performance of the proposed learning modelis superior to the classical multi-voxel pattern analysis (MVPA) methods for the underlying cognitive process.